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Abstract 

Teaching students how to think critically and develop lifelong habits of evidence-based 
inquiry outside of the classroom is a primary goal for educators today. This paper de¬ 
scribes nine activities designed to promote evidence-based critical thinking in college or 
high school classrooms in any discipline. We have developed a seven step process for 
critical thinking, with teaching modules designed to build skills in these steps in an en¬ 
gaging, active way. The modules involve a variety of teaching methods, including use of 
video, discussion, debate, and homework assignments. We begin with fun, engaging, less 
emotionally-laden topics such as toys that claim to read brain waves or pictures of ghosts 
and then progress to more serious topics such as use of medical marijuana and racial pro¬ 
filing in airports. The modules were designed to stimulate interest in our students and 
could easily be modified to encourage students to think more deeply about current issues 
in the news or local community. There is evidence that these modules can increase moti¬ 
vation to think critically outside the classroom (Burke, Sears & Kraus, 2012) and help 
students evaluate their own belief systems (Burke, Sears, Kraus, & Roberts-Cady, in 
press). Further, we report on data suggesting that, when combined with deductive reason¬ 
ing activities, these modules can boost students’ critical thinking skills. 

Keywords: Critical thinking, teaching, classroom activities, paranormal beliefs. 


Today’s students are drowning in ‘facts.’ They have information readily available at eve¬ 
ry moment on their internet-connected devices. Google and Wikipedia alone can answer 
most questions at the touch of a screen or click of a mouse. Easy access to information 
makes the memorization of basic facts—once the hallmark of education—largely irrele¬ 
vant in the modern world. The vast amount of information available calls instead for hon¬ 
ing of different skills. While students are repeatedly reminded not to believe everything 
they read or see on TV or other media devices, many still consider on-line open source 
sites to be acceptably reliable sources of information. Thus, choosing which information 
merits attention and knowing how to weigh the evidence for supposed ‘facts’ are criti¬ 
cally necessary skills for the information age. Consumers of information must be able to 
delineate between well supported claims and those that rely on ‘truthiness,’ or using a 
gut-sense feeling instead of empirical evidence or thinking to determine truth (Colbert, 
2005). Truthiness is also defined as "the quality of preferring concepts or facts one 
wishes to be true, rather than concepts or facts known to be true" (Merriam-Webster, 
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2006). A major challenge for educating young and emerging adults is helping them to 
develop critical thinking skills that translate beyond the classroom walls and will allow 
them to make informed choices based more on truth than truthiness (Paul, 2005; Wyer, 
2009). 

Critical thinking is a complex concept that has been defined in a number of ways, includ¬ 
ing as metacognition (Paul, 2005), as logical argument analysis (Watson & Glaser, 2006), 
and as careful weighing of the evidence to support a claim (Bensley, 1998). While most 
educators agree that it is vital to teach critical thinking (Flores, Matkin, Burbach, Quinn 
& Harding, 2012; Wyer, 2009), we do not always agree on the definition or specific skills 
we are hoping to instill in students (Chenault & Duclos-Orsello, 2008). With this chal¬ 
lenge in mind, we set out to create classroom modules that promote critical, empirically- 
based thinking skills. We based the modules on Bernstein’s (2007) five steps for critical 
thinking. He proposed that students needed to think about the claim (‘what am I being 
asked to believe?’), evaluate the evidence, consider alternative interpretations of the evi¬ 
dence, and, finally, draw conclusions. These steps are similar to the subtests in the Wat- 
son-Glaser critical thinking test (inference, recognition of assumptions, deduction, inter¬ 
pretation and evaluation of arguments; Watson & Glaser, 2006). In addition to these con¬ 
structs, we sought to address potential barriers to critical thinking, such as biases, emo¬ 
tional reasoning, overuse of personal experience or small case studies, and reliance on 
authority (Myers, 2009) directly in our modules. We therefore created the following sev¬ 
en steps to critical thinking as the foundation around which we then designed our class¬ 
room teaching modules. 


Critical Thinking: Seven Steps 

1) What am I being asked to believe or accept ? 

2) What evidence is available to support the claim? 

3) What alternative ways are there to interpret the evidence? 

4) Rate the evidence/alternatives on 0-10 scale based on validity/strength 

5) What assumptions or biases came up when doing the above steps? 

(e.g., using intuition/emotion, authority, or personal experience rather than 
science) 

6) What additional evidence would help us evaluate the alternatives? 

7) What conclusions are most reasonable or likely? 

We were aware that students might be initially resistant to focusing on critical thinking, 
as this type of thinking requires more cognitive effort than simply relying on authority or 
intuition (Browne & Freeman, 2000). We were thus careful in our design to choose en¬ 
gaging and timely topics as well as utilize considerable active learning to optimize stu¬ 
dent motivation. We designed nine brief critical thinking (CT) modules for use about 
once per week throughout the semester. We tested these modules in a wide variety of col¬ 
lege psychology classrooms, ranging from introductory psychology to research methods 
and senior seminar, before ultimately implementing them in a newly designed course 
called “Critical Thinking in Psychology.” The activities were structured such that, each 
week, we built upon the steps that had been the focus of the previous week’s module. For 
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example, the first activity centered on identifying claims and evidence, and the next 
module added in brainstorming about alternative interpretations of evidence. Each mod¬ 
ule featured an informational presentation and associated class activity (e.g., discussion, 
debate, or writing assignments) about a different controversial topic or issue such as med¬ 
ical marijuana, whether vaccines can cause autism, ghost photos, racial profiling, dog 
breed bans, and psychic powers. The topic areas could easily be modified to follow cur¬ 
rent debates of interest to students, but we believe that active learning is important to stu¬ 
dent engagement in the material. For development of critical thinking, our impression is 
that scaffolding the seven steps across sessions would be optimal. However, instructors 
could use and adapt individual exercises depending on the context and goals of their 
course. 

We will discuss several of these modules in depth to illustrate how we present critical 
thinking in an active way in the classroom, as well as other modules with less detail to 
provide ideas for you to build from in your own classrooms. 

The Modules 

Module 1: Star Wars Force Trainer - This module is a stimulating introduction to the 
steps of critical thinking. We bring a Star Wars Force Trainer to class ($45; Uncle Mil- 
ton, 2009). This is an educational toy that claims that by “utilizing dry neural sensor 
technology, the headset reads and interprets your brainwaves” (NeuroSky, 2011). The 
learning guide that accompanies the toy discusses various types and functions of brain 
waves and compares the toy to EEG machines that develop relaxed concentration (pre¬ 
sumably theta waves). The guide claims that the user’s relaxed brain waves cause a small 
ball to move up a tube attached to the sensor. Students read the material that comes with 
the toy, along with an article on the future of brain-controlled devices that hypothesizes 
that the future holds help for Alzheimer’s patients and kids with ADHD through these 
devices that use electrodes to monitor concentration (Hammock, 2009). The students then 
watch a short clip on YouTube to illustrate how the toy works: htt p://www.youtube.com/ 
watch?v=6MFOduNUE8U . 

We have students identify the claims the manufacturer is making about the Force Trainer. 
Encouraging brainstorming of claims before looking at evidence is an important first step 
to critical thinking, one that is often overlooked in the rush to judgment. Once we gener¬ 
ate a list of claims (such as that the machine is accurately reading and interpreting brain 
waves, and objects can be moved by developing certain brain waves), we ask students to 
test the device. This is when the fun begins. Many students do think that when they con¬ 
centrate carefully the ball is moved farther, and they begin to be convinced that the initial 
claims might be true. Testimonials from parents of autistic children who claim the force 
trainer helped their child learn to relate better with others and other YouTube evidence 
also exist to support these claims. 

We then move to step 3, generating other ways to interpret the evidence, as well as ideas 
for testing alternative hypotheses for how the Force Trainer works (which is part of step 
6). These range from trying the Force Trainer on non-animate objects (which does not 
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work, supporting the manufacturer’s claims) to trying it on dogs (which is possible but 
difficult, so we push for other ideas) to trying it on things that conduct electricity but do 
not have brain waves, like root vegetables. Left to brainstorm long enough, most groups 
develop the idea to try the force trainer on other parts of their own bodies. Not surpris¬ 
ingly, most students’ knees have the same ability (if not greater) to move the ball ‘with 
the force of theta waves’ as their heads. This is an obvious problem for the manufac¬ 
turer’s claim (we are not aware of any brain waves in our legs) and a memorable lesson 
in critical thinking. Students learn that there might be multiple explanations for the evi¬ 
dence they see with their own eyes, and may start to think that critical thinking can be fun 
and valuable. 

Module 2: Photos of Ghosts - The second module also focuses mainly on identifying 
claims and thinking of alternative explanations for the existing evidence. We start with a 
few statistics from a 2011 Rasmussen poll that suggests that 31% of adults believe in 
ghosts (Rasmussen Reports, 2011). We then show a PowerPoint slide show of supposed 
ghost pictures and have students evaluate the claims and the evidence. It is important not 
to skip the first step of evaluating claims. Many students want to simplify the claim to 
state that ghosts exist, but if prodded, they will recognize that there are more embedded 
claims, such as that ghosts can be photographed with certain technology. We then move 
to the evidence of ghosts provided by the photographs. Being skeptical of photos found 
on the web is second nature to today’s students, but we ask them to come up with other 
explanations beyond Photoshop. In one classic picture of a ghost hugging a child, for ex¬ 
ample, the ghost in question could be smoke from the photographer’s cigarette. We also 
discuss optical illusions such as the Muller-Lyer illusion where you ‘see’ something that 
does not exist by filling in missing parts of a pattern you expect to find (Muller-Lyer, 
1889). 

We end this discussion by questioning whether the alternative explanations for the photo¬ 
graphic evidence actually mean that ghosts do not exist. When we first ask what we have 
concluded, often students jump directly to ‘ghosts do not exist’ but, if questioned, they 
conclude that ghosts may or may not exist, but they cannot be photographed. Astute stu¬ 
dents will point out that we have not actually supported that claim either, and that we 
simply think that these photos are probably not of ghosts. This final discussion is most 
useful in evaluating claims, evidence, and alternatives, and is an important caution to not 
over-step one’s data. It also introduces the fact that using critical thinking does not neces¬ 
sarily mean you cannot believe in paranormal phenomenon—rather, it simply requires 
you to examine the evidence for your beliefs. As an instructor and scientist, maintaining 
this openness to possibility is important, especially early in the modules so as not to al¬ 
ienate students. 

Module 3: Astrology activity - Many students read their horoscope regularly, with some 
degree of belief in those predictions, so this module is highly relevant for them and may 
engender resistance if not handled carefully. As with each module, we ask students to 
brainstorm about the basic claim of astrology (step 1). One simple claim is that personal¬ 
ity types are associated with particular Zodiac signs. We ask students to tell us what evi¬ 
dence there is to support this claim (step 2). They generally offer personal anecdotes or 
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stories in which their horoscope was correct or where the description given based on their 
birth Zodiac sign has been accurate. Because we have already focused on step 3, alterna¬ 
tive ways to interpret evidence, in this module we spend more time discussing step 5, bi¬ 
ases and assumptions. We discuss cognitive biases such as illusory correlation (Hamilton 
& Gifford, 1976) and confirmation bias (Nickerson, 1998; Watson, 1960). We then ask 
students to brainstorm how they could use the scientific method to test the claim (step 6). 
After brainstorming about methods, we give students a handout with 12 personality pro¬ 
files that come from a book on astrology (March & McEvans, 1982). We ask them to cir¬ 
cle the description that best describes their personality. Once they have made their 
choice, we show them the “correct” answers, that is, which descriptions go with what 
birthdates according to the astrology book. We then count how many students circled the 
personality description that is supposed to correspond to their birthdate. As the “correct” 
answers are revealed, those students who have chosen the right answers rejoice, and those 
who did not have a match are usually more subdued. We can see the attraction of confir¬ 
mation bias clearly. This leads to a discussion of how many students should match before 
we are convinced that the correct answers represent more than chance. In a class of 25 
students, for example, one would expect about two or three to guess correctly if the 
choices were random since the odds are 1-in-12. We can also discuss how small numbers 
that occur naturally by chance can be over-interpreted. Students who are really thinking 
critically will point out that astrology could still be correct, but that our descriptions may 
have come from a weak source (i.e., outdated book), for which they get bonus points. We 
end with a brief review of the scientific literature that shows no empirical support for 
predicting personality using birth dates (Saklofske, Kelley, & McKerracher, 1982; Tyson, 
1980). 

Module 4: Psychic abilities - In this module, we use a short video clip of Uri Geller to 
introduce the claims of his psychic abilities (we show the first 6 minutes of 
http://www.youtube.com/watch?v=M9w7iHYriFo) . We then ask students to individually 
write down what they see as the claim (step 1), the evidence that supports the claim (step 
2), alternative explanations for the evidence (step 3), and to create a study design that 
would fairly evaluate the claim (step 6). After they have each designed a study, we show 
the students the next 7-minutes from the same video (above) that shows Uri Geller failing 
to produce results on the Tonight Show. We discuss the ironic finding that after the show 
aired, belief in psychic ability actually increased. We then visit the James Randi Educa¬ 
tional Foundation website (http://www.randi.org/site/) , which is devoted to the scientific 
study of psychic claims. We discuss their one million dollar challenge, which states that 
“the JREF will pay US$1,000,000 (One Million US Dollars) ("The Prize") to any person 
who demonstrates any psychic, supernatural, or paranormal ability under satisfactory ob¬ 
servation.” (JREF, 2012). This challenge has existed since 1964, and well over a thou¬ 
sand applicants have tried to win the prize. To date, not one single person has been able 
to prove their psychic abilities in a scientific test. This website also provides interesting 
examples of tests designed by the Foundation. 

Module 5: Pit bull Ban - In module 5, we start with a newspaper article about banning 
pit bull dog breeds in Denver, CO (Kass, 2005). As in the previous module, we have stu¬ 
dents write individual answers for step 1, the claim, and step 2, the evidence provided. 
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We then focus squarely on step 4, evaluating the existing evidence. We write all the evi¬ 
dence from the article on the board and have students rank order each statement from 
most convincing to least. This ranking process, done in discussion with peers, is an effec¬ 
tive way to get students thinking about what constitutes compelling evidence and why. 
The goal for this module is not to come to a conclusion about the pit bull ban, but rather 
to recognize that we do not yet have enough information to make a decision. We end with 
a discussion of what additional evidence would be needed to make a fair decision about 
whether pit bulls should be banned. Note that this module can easily be modified to focus 
on virtually any currently newsworthy event (e.g., did the FBI or CIA miss something 
years earlier in the case of the Boston Bombers?). 

Module 6: Deal or No Deal - In this module, we focus primarily on step 5, the biases 
that come onto play when making decisions. We emphasize metacognition here, which is 
thinking about your thinking. We begin with a clip of the TV game show Deal or No 
Deal at http://www.youtube.com/watch?v=hmZFHi Qfx-o and ask students to think about 
biases that might come into play and lead the contestant to make decisions that they later 
regret. Screening in over 40 countries, Deal or No Deal became an international televi¬ 
sion sensation in the 21st century (Deal or No Deal Countries, 2012). Once students are 
introduced to the game, we show a clip of a contestant who was offered $603,000.00 and 
ended up with $1.00 (http://www.youtube.com/watch?v=MQ40bwT-OfU) , directing stu¬ 
dents to pay particular attention to the biases in the advice given to the contestant. We 
then have students make a plan for how they would make decisions in this game, and ask 
a volunteer to play the game while other students give them (hopefully) solid advice 
(http://www.nbc.com/Deal or No Deal/game/flash.shtml) . This is an excellent exercise 
to show how difficult it is to stick to a rational plan in the face of high emotions and peer 
pressure. 

A scientific study of the show (Post, van den Assem, Baltussen, & Thaler, 2008) found 
that several cognitive heuristics come into play that can explain contestants’ decisions. 
Notably, the break-even effect causes losers to take greater risks due to incomplete adap¬ 
tation to prior losses, and the house-money effect leads contestants who do well in early 
rounds to make riskier decisions later because the money they currently hold does not 
seem like it is theirs. Ironically, risky decisions in this instance lead to both the biggest 
losses AND the biggest winnings in Deal or No Deal, while rational strategies typically 
yield more moderate amounts of prize money. This interactive experience in decision¬ 
making may help students identify the pressures that could lead to poor choices in other 
life situations. Social pressure to stay at a party and drink, for example, often sways stu¬ 
dents who have rational plans to get a good night’s sleep or study. Students can generate 
their own examples of situations in which critical decision-making would be valuable. 

Module 7: Autism and Vaccines - This module is similar to module 4 on psychic phe¬ 
nomena in that we start with a video presentation and evaluate the evidence for the 
claims. However, in this instance, we are focusing on real-world problems and families 
who are making life and death decisions with high emotional load. We show a CBS news 
segment about a court case in which Michael and Theresa Cedillo tried (and failed) to 
prove that vaccines were responsible for their child’s severe autism 
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(http://www.youtube.com/watch?v=rOGOkSOuXWE) . The video provides a forum for 
both sides to present their evidence. Students are asked to pay close attention to the types 
of evidence presented by each side, and weigh strengths and weaknesses of these argu¬ 
ments (step 4). Students then draw conclusions and explain what evidence they used to 
reach those conclusions (step 7). The emotional component of the mother’s grief over her 
child’s condition is also discussed, and links can be made to the student’s own experience 
with Deal or No Deal. With more class time, instructors could also show Jenny McCar¬ 
thy and Jim Carrey, two popular celebrities, discussing their beliefs about the vaccine- 
autism link on Larry King Live at htt p://www.youtube.com/watch?v=HX-SCdiDOrA ; 
despite the lack of scientific basis for their beliefs, these movie stars have convinced 
many parents to forego essential vaccinations for their children (step 5). Almost one quar¬ 
ter of parents currently believe that vaccines might be dangerous, and, accordingly, child 
vaccination rates are declining at a rate of 3-4% per year (Nixon, 2010). We invite stu¬ 
dents to think about what information they would need to make a sound decision about 
vaccination for their own families, and, finally, we present scientific evidence that chil¬ 
dren who are vaccinated tend to have lower rates of developmental disorders, including 
autism (Andrews et al., 2004). 

Modules 8 & 9: Topics in the news: medical marijuana and profiling - These last two 
modules will be considered together, as they each involve debate methods and are de¬ 
signed to get students thinking about real-world controversies and social issues using our 
seven-step method. The medical marijuana issue is an examination of a recent decision 
by our college campus to ban the use of medical marijuana, which is legal in our state. 
We divide the students into two groups based on their initial leaning for or against the 
ban. Students are assigned to argue the alternate point of view from their own initial reac¬ 
tion. Those who do not have strong feelings pro or con are divided in such a way as to 
balance group size. The homework is for all students to bring in at least three pieces of 
evidence for their assigned side of the debate (step 2). We then have an in-class debate in 
which each side presents its case, uninterrupted, using their best evidence. After each 
group has presented their case, they may directly question each other. Following the de¬ 
bate, students are then asked to write an individual essay on their own beliefs, and sup¬ 
port their view with evidence (steps 4 and 7). Many mention that arguing a point they did 
not originally believe caused them to look more closely at the evidence, and many either 
changed their view, or became more open to the other side’s argument. The goal was 
clearly to formulate an informed decision for themselves while being mindful of the evi¬ 
dence used to form this opinion. 

The module about racial profiling took the process a step farther. We examined evidence 
for and against racial profiling, beginning with a discussion of current use of racial profil¬ 
ing by airline security, and expert views on profiling by TSA (Press, 2009). We then dis¬ 
cussed the case of racial profiling by Maryland State Police (ACLU, 2010) as well as lo¬ 
cal profiling by looking at ads in our city newspaper that list “no dogs, no smokers, no 
students.” Many students have had experiences of similar discrimination, because many 
landlords believe that students might in fact be worse renters on average. This leads to a 
lively discussion of when/if scientific evidence trumps moral reasoning. Is it right to pro¬ 
file in the name of public safety? Should your 80 year-old grandmother be searched in the 
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airport as often as a strong young man? Do landlords have the right to only rent to people 
over 30 years old or people with full time jobs? It is important to note that critical think¬ 
ing is one way to answer these questions, but social justice and morality might also be a 
necessary part of the equation. 

Empirical Support for the Modules 

We examined the effectiveness of these modules with 128 college students and found that 
they encourage students to use critical thinking more in their daily lives and to critically 
evaluate their own beliefs, particularly about paranormal phenomena (Burke, Sears, 
Kraus & Roberts-Cady, in press). Although our modules significantly reduced paranor¬ 
mal beliefs from pre- to post-semester testing, they did not, when used by themselves, 
change scores on the Watson-Glaser Critical Thinking Appraisal, Form S (WGCTA-FS; 
Watson & Glaser, 2006), which is primarily a test of deductive reasoning. For an in-depth 
discussion of these findings and a critique of the current literature examining testing of 
critical thinking, see Burke, Sears, Kraus & Roberts-Cady (in press). 

Because our initial study did not show increases in deductive reasoning skills, in the 
Spring of 2013, the third author (BLB) modified his “Critical Thinking in Psychology” 
(CT) course to include 10 minutes per week of deductive reasoning practice along with 
several of the modules described above and some new ones along student interest (e.g., 
the value of a college degree, critical thinking of religion). The deductive reasoning prac¬ 
tice used problems similar to the Watson-Glaser test although with psychology examples. 
These problems typically present a short statement with a variety of possible conclusions. 
Test takers are asked to evaluate the strength of each conclusion and identify assumptions 
that appear to have been made. Students completed problems individually, and then dis¬ 
cussed their answers in small groups, explaining their reasoning to their peers. The entire 
class later discussed the correct answers. 

In this current study, we compared the CT class (n=20) results to those of an introductory 
math class, which was used as a control group (n=19). Pre and post semester testing in¬ 
cluded The Revised Paranormal Belief Scale (RPBS; Tobacyk, 2004; Tobacyk & Mil¬ 
ford, 1983) and the WGCTA-FS test of critical thinking skills (Watson & Glaser, 2006). 

Mixed model 2(pre/post) X 2(CT/math) ANOVAs with alpha set at .05 were used to ex¬ 
amine results. As expected from previous studies of our modules, we found a significant 
interaction between pre-post measures of paranormal belief and class, F (\, 35)=9.60, 
p=.004, rf=. 215. The math group had pre and post test scores of 84.76 (577=20.86) and 
84.97 (527=20.91) respectively. The CT class had pretest scores similar to the math stu¬ 
dents with an average of 81.85 (577=21.89). but a significantly lower posttest average of 
65.85 (527=20.29). 

Critical thinking scores also showed significant interactions, F(l, 32)=5.03, p=. 03, 
?7'=.136. The math scores on the WGCTA-FS were virtually identical throughout the se¬ 
mester, averaging 21.50 (527=2.96) at pretest and 21.29 (527=4.53) at posttest. The CT 
students started higher, perhaps because they are more advanced students. Their pretest 
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average was 28.10 (50=5.87) but they also improved significantly over the semester, 
with a posttest average of 32.30 (50=5.51). 

Although these results are preliminary in nature, they show clear promise for our method 
of teaching critical thinking. It is interesting to note that the students who had the deduc¬ 
tive reasoning with the modules increased their CT scores by an average of 17%, while in 
our previous study, philosophy students, who were trained in deductive reasoning without 
use of the modules, increased only 8% on average (Burke, Sears, Kraus & Roberts-Cady, 
in press). It is therefore possible that the active, engaged learning promoted by the mod¬ 
ules is useful above and beyond standard deductive logic skill training and may be opti¬ 
mal when combined with them. 


Conclusions 

These modules are suggestions for how to get students to exercise their critical thinking 
muscles. They can be used individually or as a series of building modules in almost any 
class—psychology or beyond—that has critical thinking as one of its goals. Each could 
be adapted to fit the interests of your students and hot topics of the day or of your 
city/campus. It is our hope that the descriptions of these modules herein will spur teach¬ 
ers into creating their own interactive ways to foster more critical thinking in the class¬ 
room. Many researchers argue that critical thinking is a vital life skill and lament the lack 
of effective critical thinking training in higher education (Flores, Matkin, Burbach, Quinn 
& Harding, 2012; Paul, 2005; Wyer, 2009). Our modules contain the key features pro¬ 
posed by Browne and Freeman (2000) for critical thinking classrooms: active learning, 
developmental tension, and fascination with the contingency of conclusions. 

Recent reports suggest that many people are using YouTube as a daily source of news, 
and that reliance on sources that have no established standards for accuracy is growing 
rapidly (Pew, 2012). Clearly, students (and society) would benefit from more practice at 
looking deeper than the surface ‘truthiness’ of information. Our hope is that this habit of 
mind will become engrained with repeated practice, and will be used in everyday life 
such as medical decisions, better informed consumer choices, and political decisions. If 
educators work together toward this goal, we can encourage a generation of students who 
know how to think for themselves and do not simply believe whatever they read or see on 
the internet. 
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